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WASHINGTON  D  C  20310 


DACS-DMO  10  October  1984 

SUBJECT:  Army  Study  Highlights 


SEE  DISTRIBUTION 


1.  Volume  V  of  the  Army  Study  Highlights  Report  presents  the 
5th  set  of  Gists  of  five  high  quality  studies  recently  selected 
for  publication  by  members  of  the  Army  study  community. 

2.  The  report  also  contains  a  summary  of  lessons  learned  from 
peer  reviews  of  a  small  sample  of  recent  studies  and  a  section  on 
Review  of  Army  Analysis  Extended  (RAAEX).  The  purpose  of  the 
study  highlights  reports  has  thus  changed  somewhat.  In  addition 
to  its  use  as  a  way  to  widely  publicize  high  quality  studies, 
encourage  excellence  in  Army  analysis  activities  and  give 
visibility  to  deserving  individual  analysts,  the  report  will  also 
inform  the  analysis  community  about  matters  of  interest  to  that 
community.  We  appreciate  your  comments  on  the  expanded  role  for 
the  report  as  well  as  suggestions  for  items  of  interest  for  future 
editions. 

3.  Wide  distribution  should  be  made  throughout  your 
organizations.  The  selected  studies  are  examples  of  efforts  which 
will  enhance  the  Army  goals  and  missions.  The  lessons  learned 
information  is  valuable  to  both  analysts  and  sponsors.  The 

RAAEX  recommendations  will  have  a  far-reaching  impact  upon  the 
future  quality  and  conduct  of  studies  and  analysis  in  the  Army. 

4.  Comments  about  the  report  and  suggestions  should  be  made  to 
Ms.  Gloria  Brown  of  this  office,  at  AV  227-0026  or  (C)  (202)  697- 
0026.  Ms.  Brown  can  also  provide  additional  copies  on  request. 


JOANN  H.  LANGSTON,  Director 
Study  Program  Management  Office 
Management  Directorate 
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ONE  SHEET 

V  ••  •  USACE  MOBILIZATION  POSTURE 

*  *■*•*■* 

■  ■  ■  - - —  ■  ■  - - 

STUDY  GIST 

- - 

THE  PRINCIPAL  FINDINGS  of  this  study  were: 

(1)  USACE  does  not  provide  adequate  individual  training  in  mobilization 
roles  and  missions. 


(2)  Most  USACE  managers  are  unsure  about  how  to  remove  construction 
constraints  under  emergency  conditions. 

(3)  Mobilization  planning  is  not  well  coordinated  among  other  Federal 
and  non-Federal  agencies,  particularly  at  the  regional  level. 

(4)  Mobilization  staffing,  material  resources,  and  funding  are  not  well 
provided  for. 

(5)  Mobilization  requirements  are  not  clearly  defined. 

(6)  The  distinction  between  full  and  total  mobilization  is  unclear. 

(7)  The  Direct  Support/General  Support  District  concept  may  be 
inef  fective. 

(8)  Existing  and  planned  ADP  systems  are  not  designed  to  operate  effec¬ 
tively  under  mobilization  conditions. 

(9)  Plans  for  maintaining  the  physical  security  of  USACE  facilities 
during  a  mobilization  are  inadequate. 

(10)  There  is  a  perception  that  USACE  does  not  provide  enough  command 
emphasis  and  guidance  for  mobilization  planning. 

THE  MAIN  ASSUMPTIONS  on  which  this  study  is  based  are: 

(1)  In  a  national  mobilization,  USACE  would  be  the  primary  agency 
expected  to  meet  the  construction  and  emergency  response  needs  of  the  military 
and  local  governments. 

(2)  The  primary  responsibility  for  USACE's  mobilization  support  would 
fall  to  USACE's  divisions  and  districts. 

(3)  USACE  is  primarily  structured  for  its  peacetime  role,  not  for  a 
mobilization. 

THE  PRINCIPAL  LIMITATIONS  of  this  study  which  may  affect  the  findings  are:  No 
mobilization  exercise  has  tested  USACE's  ability  to  transition  from  full  to 
total  mobilization.  Therefore,  it  has  not  been  possible  to  assess  the  entire 
spectrum  of  requirements  that  could  be  levied  against  USACE  FOAs  during  a 


total  mobilization,  or  the  potential  impact  a  total  mobilization  could  have  on 
USACE  operations. 


THE  SCOPE  OF  THE  STUDY.  This  study: 

(1)  Evaluates  the  current  USACE  mobilization  posture  for  full  and  total 
mobilization. 

(2)  Outlines  a  conceptual  mobilization  posture  for  USACE,  and  identifies 
the  principles  and  limitations  which  form  the  framework  within  which  USACE 
must  do  mobilization  planning. 

(3)  Presents  a  program  that  USACE  should  follow  over  the  next  5  to  8 
years  to  further  improve  its  mobilization  readiness  posture. 

THE  STUDY  OBJECTIVE. 


The  objective  of  this  study  was  to  develop  a  program  USACE  should  follow 
to  improve  its  posture  for  supporting  full  and  total  mobilization. 

THE  BASIC  APPROACH. 

(1)  The  study  first  evaluated  USACE  mobilization  requirements  and  capa¬ 
bilities  as  assessed  from  MOBEX-78,  80,  and  83  to  determine  the  changes  in  the 
USACE  mobilization  posture  over  that  time  period. 

(2)  The  study  next  outlined  a  conceptual  framework  for  a  mobilization 
posture  that  USACE  should  achieve  if  it  is  to  effectively  support  either  full 
or  total  mobilization.  This  conceptual  framework  was  developed  in  considera¬ 
tion  of  the  principles  set  forth  by  the  national  leaders,  the  Army  Chiefs  of 
Staff,  and  past  and  present  Chiefs  of  Engineers.  This  conceptual  mobilization 
posture  was  then  suppressed,  based  on  a  number  of  "real  world"  limitations,  to 
produce  a  "reasonably  attainable"  mobilization  posture. 

(3)  Finally,  the  study  sets  forth  a  5-  to  8-year  program  for  USACE  to 

follow  in  enhancing  its  mobilization  posture.  The  study  provides  two  sets 

of  recommendations:  one  set  for  USACE-wide  implementation  and  a  second  set 
for  HQ  USACE  Implementation.  Additionally,  the  study  provides  schedules  for 
mobilization  activities  and  the  implementation  of  the  study  recommendations 
over  the  FY  84-90  period. 

THE  REASONS  FOR  PERFORMING  THE  STUDY. 

MG  E.R.  Heiberg  III,  Deputy  Commander,  USACE,  requested  (in  a  30  August 
1982  DF)  that  ESC  conduct  a  study  evaluating  USACE's  performance  during  MOBEX- 
83,  assess  USACE's  mobilization  posture,  and  develop  a  mobilization  improve¬ 
ment  program  for  USACE. 

DT1C  ACCESSION  NUMBERS  OF  THE  FINAL  REPORTS  ARE  B077451L,  B079267L,  AND 

B082221L. 

COMMENTS  AND  QUESTIONS  MAY  BE  SENT  TO: 


Engineer  Studies  Center 

Casey  Building  2594 

Fort  Bel voir,  VA  22060-5583 


POC:  Mr.  James  Tate 
AV:  345-2128 


THE  PRINCIFAL  FINDINGS  of  the  study  are: 


(1)  Ports  designated  for  receiving  military  cargo  should  be  rank  ordered  based 
on  port  vulnerability  and  operational,  multiple  vessel  support  system  (VSS),  and 
safety  capabilities.  Current  OPLANs  should  be  changed  to  reflect  this  rank  order. 

(2)  Korean  ports  have  excellent  berthing  capabilities  for  most  vessels  in  the 
MSC-control led  fleet,  NDRF  and  RRF.  However,  most  ports  lack  stern-ramp  RORO 
facilities  and  truck  and  rail  end-leading  ramps  for  expedient  intramodal  transfer. 

(3)  Many  key  ports  in  existing  OPLANs  are  capable  of  handling  military  cargo, 
but  lack  rail  service  to  their  facilities.  Since  most  rail  service  terminates  at 
locations  other  than  the  port  complexes,  multiple  handling  of  cargo  will  be  required 

(4)  Privately  owned  ports  in  Korea  have  some  of  the  best  facilities  for 
accommodating  typical  US-flag  fleet  ships  carrying  military  cargo. 

(5)  Implementation  of  the  recommendations  made  in  this  study  will  improve  the 
transportation  supportabil ity  of  the  Korean  contingency  CINCPAC  OPLAN. 

THE  MAIN  ASSUMPTIONS  of  this  study  are: 

(1)  A  combination  of  typical  US-flag  and  Korean-flag  vessel  characteristics, 
by  type  vessel,  is  an  adequate  basis  for  determining  berthing  capabilities. 

(2)  A  combination  of  port  vulnerability,  ideal  berth  factors;  and  operational, 
safety,  and  multiple  VSS  capabilities  is  an  adequate  basis  for  determining  the 
adequacy  of  a  port  for  handling  military  cargo. 

THE  PRINCIPAL  LIMITATIONS  of  this  study  are: 

(1)  Information  on  material  handling  equipment,  channel  and  pierside 
drafts,  supplied  by  the  Korean  Maritime  and  Port  Administration  District  Offices, 
could  not  be  verified  by  project  personnel. 

(2)  Port  development  projects  may  not  be  funded  and  constructed  as  planned. 

THE  SCOPE  OF  THE  STUDY  is  as  follows: 

(1)  On-site  surveys  of  22  ports  throughout  Korea. 

(2)  Development  of  an  individual  berth  rank-ordering  scheme  for  breakbulk, 
RORO,  container,  and  barge  operations. 

(3)  Development  of  an  overall  port  rank-ordering  procedure  based  on  the 
capability  of  the  ports  to  handle  unit  equipment,  general  resupply,  or  ammunition 
type  cargo  individually  or  jointly. 


(4)  Comparison  of  US-flag  and  Korean-flag  ship  characteristics  and  requirements 
with  the  berthing  characteristics  and  capabilities  of  each  surveyed  port. 

(5)  Development  of  specific  plans  that  US  and  Korean  military  and  government 
officials  should  undertake  to  offset  shortfalls  in  the  transportation  feasibility  of 
the  OPLAN. 

THE  STUDY  OBJECTIVES  were  to: 

(1)  Provide  an  up-to-date,  easily  used  reference  of  Korean  port  characteristics 
for  discharging  typical  military  cargo. 

(2)  Establish  a  priority  listing  of  ports,  by  operation  type,  best  suited  for 
deployment  and  resupply  requirements. 

(3)  Identify  ports  and  berths  suitable  for  use  as  alternatives. 

THE  BASIC  APPROACH:  Data  on  Korean  ports  were  taken  from  all  available  sources  and 

also  during  onsite  surveys.  These  data  were  then  compared,  and  needed  changes  were 
made.  The  data  were  organized  into  a  standard  format.  Ports  were  ranked  by 
commodity  type  based  on  the  newly  developed  individual  berth  and  port  rank-ordering 
schemes.  Shortfalls  in  port  clearance  capabilities  were  identified.  Recommendations 
were  made  for  sustaining  military  cargo  discharge  and  clearance  operations  during 
hostilities  against  South  Korea. 

REASONS  FOR  PERFORMING  THE  STUDY: 

In  December  1982,  the  commander,  MTMC  Terminal,  Pusan,  requested  that  MTMCTEA 
conduct  a  study  of  Korean  ports  to  determine  safe  alternative  locations  to 

discharging  ammunition.  During  the  Phase  1  OPLAN  Refinement  Conference  at  the  Joint 

Deployment  Agency,  in  January  1983,  other  significant  transportation  shortfalls  were 
identified;  the  main  one  was  lack  of  container  reception  capability.  In  March  1983, 
the  J4  EUSA/USFK,  requested  that  MTMCTEA  conduct  an  overall  origin-destination 
analysis  of  the  logistic  pipeline  that  would  be  support  Korea  during  hostilities. 
Volume  II,  The  Ports  of  Korea,  Part  1,  of  the  Korean  Ports  and  Transportation  Systems 
Capability  Study  fulfills  these  requirements.  No  comparable  study  presently  exists. 

DTIC  ACCESSION  NUMBER  OF  THE  FINAL  REPORT:  B082657 

STUDY  SPONSORS:  COMMENTS  AMD  QUESTIONS  may  be  sent  to: 

EUSA/USFK,  CINCPAC,  Military  Traffic  Management  Command 

J4-0JCS,  JDA  and  JDSSC  Transportation  Engineering  Agency 

ATTN:  MTT-TEP  (Mr.  Cooper) 

PO  Box  6276 

Newport  News,  VA  23606-0276 

AUT0V0N  927-4641 
COMMERCIAL  (804)  595-9031 

PERFORMING  ORGANIZATION  AND  AUTHORS: 

Military  Traffic  Management  Command 
Transportation  Engineering  Agency 
Will iam  J.  Cooper 
Allen  L  Snyder,  P.E. 

John  McClaire,  CPT,  TC 
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UNCLASSIFIED 


RESOURCE  CONSTRAINED  PROCUREMENT 

ONE  SHEET 

'  CAA ' 

OBJECTIVES  FOR  MUNITIONS 

STUDY  GIST 

(RECPOM)  STUDY  (U) 

CAA-SR-84-7 

(U)  THE  PRINCIPAL  FINDINGS  of  the  work  reported  herein  are  as  follows: 

(1)  (U)  Based  on  the  validation  comparison  with  P88-Europe  results 
and  the  demonstration  and  sensitivity  results,  the  RECPOM  Model  produces 
logical  responses  to  realistic  and  varied  program  and  budget  changes,  thus 
adequately  representing  a  sound  decisionmaking  process. 

(2)  (U)  The  RECPOM  Model  can  provide  a  cost  effective  mix,  allocation 
and  production  schedule  for  up  to  40  munitions  and  22  requirement 
priorities  when  changes  occur  in  funding,  priority,  production  rates,  or 
distribution  of  existing  stockpiles  as  well  as  when  munition  tradeoffs  are 
being  considered. 

(3)  (U)  A  typical  response  time  to  queries  not  requiring  model 
restructuring  or  major  data  base  changes  is  less  than  2  weeks. 

(4)  (U)  The  RECPOM  Model  cannot  currently  accept  manual  munition  al¬ 
location,  vary  munition  average  unit  cost  with  changes  in  production  quan¬ 
tity,  or  reconcile  the  effects  of  not  killing  all  required  enemy  targets. 


(U)  THE  MAIN  ASSUMPTIONS  on  which  the  work  reported  herein  rests  are  as 
follows: 

(1)  (U)  The  Army  will  continue  to  require  more  munitions  than  it  can 
afford. 

(2)  (U)  The  nature  of  the  Planning,  Programing,  Budgeting  and  Execu¬ 
tion  System  (PPBES)  process  will  continue  to  require  numerous  conditional 
examinations  of  alternative  munition  procurements  with  limited  time  avail¬ 
able  for  substantial  analysis. 


(U)  THE  PRINCIPAL  LIMITATIONS  of  this  work  which  may  affect  the  findings 
are  as  follows: 

(1)  (U)  Munition  transportation  and  storage  constraints  are  not  ad¬ 
dressed. 

(2)  (U)  The  impact  of  funding  and  other  resource  changes  is  oriented 
toward  POM  development  and  analysis  and  thus  is  shown  relative  to  the  pro¬ 
gram  force  only. 


UNCLASSIFIED 


UNCLASSIFIED 


(3)  (U)  For  the  demonstration  phase,  a  selected  set  of  high-dollar, 
high-visibility  munitions  has  been  considered  representative  of  Army  muni 
tions  over  the  FY  84-88  program  period. 


(»)  THE  SCOPE  OF  THE  STUDY  was  taken  to  include  that  the  methodology  devel¬ 
oped  be  sufficiently  flexible  to  address  program  or  budget  issues  and  that 
it  be  suitable  for  quick  turnaround  analysis. 


(U)  THE  STUDY  OBJECTIVES  were: 

(1)  (U)  To  develop  a  methodology  for  determining  within  constrained 
funding  and  production  levels  the  most  effective  mix  and  quantity  of  con¬ 
ventional  munitions  to  support  the  total  Army  worldwide  requirement. 

(2)  (U)  To  demonstrate  the  methodology  for  assumed  constrained  fund¬ 
ing  levels  using  data  from  the  P88  Europe  Study  as  a  baseline. 


(U)  THE  BASIC  APPROACH  followed  in  doing  this  study  can  be  described  as 
fol lows:  A  set  of  war  reserve,  training,  and  test  munition  goals  is 
prioritized  consistent  with  Army  force  packaging  priorities  and  established 
munition  requirements.  The  methodology  centers  around  a  goal  program 
allocation  model  which  satisfies  the  prioritized  goals  considering  munition 
unit  cost  and  consistent  with  program-budget  fund  limitations  and  munition 
production  capacity. 


(U)  THE  REASONS  FOR  PERFORMING  THE  STUDY  are  mainly  as  follows:  This  study 
responds  to  a  need  for  an  expeditious  method  to  aid  in  munition  allocation 
decisions  and  an  assessment  of  their  impact  because  of  frequently  changing 
priority  and  funding  guidance  during  the  PPBES  cycle. 


(U)  THE  STUDY  SPONSOR  was  the  Deputy  Chief  of  Staff  for  Operations  and 
Plans  who  sponsored  the  work,  established  the  objectives,  and  monitored 
study  activities. 


(U)  THE  STUDY  EFFORT  was  directed  by  Mr.  Ronald  J.  Iekel,  Requirements  and 
Resources  Directorate,  CAA. 


(U)  COWENTS  AND  QUESTIONS  may  be  sent  to  the  Assistant  Director  for 
Requirements  and  Resources,  US  Army  Concepts  Analysis  Agency,  8120  Woodmont 
Avenue,  Bethesda,  MD  20814,  AUT0V0N  295-5251. 

OTIC  number  C034682. 
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US  ARMY  MATERIEL  SYSTEMS  ANALYSIS  ACTIVITY 


SOURCES  AND  IMPACTS  OF  INSTABILITY  IN  WEAPON 

SYSTEMS  ACQUISITION 


1.  iftCKGKOUNU  AMP  MOSLEM  S'lATSrtvVr:  TWo  high  level  Army  Lask  forces  (the 
Acquisition  Steering  Group  and  Che  Cost  Discipline  Advisory  Coinmi  ttee)  concluded 
that  program  instability  is  a  major  cause  of  both  cost  and  schedule  growth. 

Ibis  view  was  corroborated  by  our  survey  of  project  managers,  who  identified 
requirements,  funding,  technological  and  personnel  instabilities  as  significant 
impediments  to  program  success.  Subsequent  research  focused  on  ways  to  better 
•Linage  these  four  instability  types. 

2.  05 JILT IVES:  The  objectives  of  this  study  were  to  (i)  identify  instability 
factors  vjhich  affect  Army  programs;  (ii)  isolate  instability  factors  which  the 
Army  can  control;  (iii)  analyze  the  impact  of  each  Army-controllable  instability 
factor;  (iv)  assess  proper  balance  of  stability  and  flexibility;  (v)  determine 
the  source  of  each  Army -controllable,  undesirable  instability  factor;  (vi) 
examine  current  stabilization  initiatives;  and  (vii)  recommend  new  or  revised 
initiatives  to  attain  the  proper  stability/ flexibility  balance. 

i.  RESEARCH  DESIGN:  Research  efforts  consisted  of  (i)  an  in-depth  Literature 
review;  (ii)  a  survey  of  past  and  present  project  managers;  (iii)  an  analysis  of 
current  initiatives;  (iv)  interviews  with  Army  resource  and  acquisition 
management  personnel;  and  (v)  synthesis  of  results. 

4.  001111113 IONS:  (i)  instability  pervades  the  Army's  requisition  and  resource 
managenent  processes  and  is  nost  pronounced  in  requirements ,  technology , 
funding  and  personnel  practices;  (Li)  the  Armv,  itself,  Is  a  key  contributor 
to  program  instability;  (iii)  current  procedural  initiatives  address  some,  but 
not  all,  of  the  project  managers’  concern ;  .uni  (iv)  more  basic  reforms  will 
be  needed  to  effect  the  proper  stability  '  t  lexibi  1  rv  balance. 

5.  KHIONM'INDA I  IONS :  A  threefold  approach  U  >  ;>r  >gr.an  stabilization  is 

recoinnended :  (i)  introduction  of  nutria  :iui  lgeneot  for  the  Arny  General  Staff 

to  provide  better  mission  focus  and  fun iot,.»l  integration;  (ii)  review  of 
personnel  practices  to  include  stifling,  promotion  and  rotation  policies; 

(iii)  tenacious  implementation  of  carreit  ini^i  r  ives  with  emphasis  on 
problems  surfaced  by  our  survey  and  ex;vinded  use  of  mulriyear  procurement. 

6.  COMENTS  AND  QUESTIONS  FAY  HE  SElJi  L'U:  Mr.  Duane  Knittle  or  Mr.  Charles 
Smith,  US  Arny  Procurement  Research  Office,  Ar.iy  Materiel  Systems  Analysis 
Activity,  Fort  Lee,  VA  23801-6046  AUIQVON  687-4381 

7.  iafUDY  SPONSOR:  DA,  Office  of  the  Comptroller  of  the  Arny,  DACA-CAZ-A. 

8.  STUDY  EFFORT  WAS  DIRECTED  6Y:  HQ,  [>\RC0M,  DRCPP-S. 


9.  DT1C  ACCESSION  NIM1ER  CF  FINAL  DD1498:  [YA301430. 


ONE  SHEET 
STUDY  GIST 


THE  INTERNATIONAL  ECONOMIC,  READINESS, 
AND  LOGISTIC  SUPPORT  COST 
MODEL:  CONUS  VS  OCONUS  SITES  FOR 
DEPOT  LEVEL  MAINTENANCE 
(COST  COMPARISON  MODEL) 


THE  PRINCIPAL  FINDINGS 

(1)  The  net  costs  to  the  U.S.  Government  of  performing  depot  maintenance 
(and  therefore  many  other  military  functions)  at  CONUS  sites  versus  OCONUS 
sites  is  significantly  less  than  many  previous  studies  have  shown. 

(2)  The  primary  factor  in  determining  if  it  is  cost  effective  to  overhaul 
an  item  in  CONUS  vs  OCONUS  is  manhours  per  item  required  divided  by  measurement 
tons  (shipping  cubic  feet/40)  per  item. 

(3)  Other  major  factors  are  the  DM/ $1  exchange  rate,  the  total  U.  S.  tax 
rate,  the  additional  facilities  and  equipment  costs,  and  annual  replacement  pipe- 
1 ine  costs . 

THE  MAIN  ASSUMPTIONS 

(1)  CONUS  workers  pay  local,  state  and  federal  U.  S.  taxes  while  foreign 
workers  at  OCONUS  sites  do  not. 

(2)  Each  new  or  additional  CONUS  job  creates  other  CONUS  support  jobs 
which  also  pay  U.  S.  taxes. 

(3)  Some,  but  not  nearly  all,  of  the  new  CONUS  jobs  would  be  filled  by 
people  currently  drawing  unemployment  and/or  welfare  benefits. 

THE  PRINCIPAL  LIMITATIONS 

(1)  Limited  data  is  available  on  the  percentage  of  people  hired  for  depot 
type  jobs  who  would  be  drawing  each  type  of  unemployment  and/or  welfare 
benefits. 

(2)  Limited  data  is  available  on  how  additional  CONUS  jobs  would  increase 
indirect  taxes,  such  as  amusement,  airlines,  liquor,  business,  etc. 

SCOPE  OF  THE  STUDY 


The  scope  of  the  study  was  to  develop  a  model  to  accurately  compare  the 
cost  of  performing  depot  level  maintenance  in  the  U.  S.  versus  Europe. 

STUDY  OBJECTIVES 


(1)  Review  CONUS  vs  OCONUS  cost  comparison  models  in  existence. 

(2)  Improve  these  models  as  needed. 

(3)  Especially  consider  the  impacts  of  CONUS  jobs. 


THE  BASIC  APPROACH 


This  model  sums  the  present  value  of  10  years  of  net  maintenance  labor 
costs,  net  packing  and  transportation  costs,  net  additional  facility 
equipment,  and  training  costs,  and  net  increased  pipeline  costs  to  maintain 
the  same  level  of  readiness  for  each  alternate  site.  The  net  cost  is  the 
actual  estimated  cost  minus  the  federal,  state,  and  local  income  taxes  and 
the  reduction  in  unemployment  and  welfare  co:  resulting  from  the  direct 

and  indirect  jobs  resulting  from  this  expenditure.  The  Construction 
Engineering  Research  Lab  economic  models  were  used  with  the  Department  of 
Commerce  U.  S.  input-output  data  to  provide  employment  and  income  multipliers 
as  well  as  income  and  employment  factors  per  $1,000,000  of  expenditures  for 
several  major  business  sectors. 

REASON  FOR  PERFORMING  THE  STUDY 

General  Thompson,  Commander,  AMC,  asked  for  such  a  study/model  when  he 
was  the  DCSL0G. 

STUDY  SPONSOR 


General  Thompson,  Commander,  AMC,  asked  for  such  a  study/model  when  he 
was  the  DCSL0G. 

STUDY  EFFORT 


Principal  Investigator: 

Carl  L.  Barton 

DESC0M  Systems  Analysis  and  Evaluation  Office 
COMMENTS  AND  QUESTIONS 


US  Army  Depot  System  Command 

Systems  Analysis  and  Evaluation  Office 

AMSDS-X 

AV  238-7232/6487 

DTIC  ACCESSION  NUMBER  OF  FINAL  REPORT  (  DA  304554  ) 
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LESSONS  LEARNED  FROM  RECENT  PEER  REVIEWS 


INTRODUCTION 

The  Deputy  Under  Secretary  of  the  Army  for  Operations  Research  has 
been  conducting  peer  reviews  of  selected  completed  studies  over 
the  past  two  years.  Although  only  a  few  studies  and  study 
agencies  have  been  reviewed,  some  trends  are  clear.  The  lessons 
learned  are  presented  to  assist  study  sponsors  and  study 
performers  in  their  search  for  ways  to  improve  the  quality  of 
studies.  Comments  on  the  lessons  learned  should  be  forwarded  to 
the  Study  Program  Management  Office  for  further  sharing  among 
members  of  the  study  community. 


LESSONS  LEARNED 

The  frequency  of  errors,  inconsistencies,  inadequacy  of 
explanation,  omission  of  apparently  important  factors,  and  poor 
presentation  indicates  a  need  to  establish  or  improve  pre¬ 
publication  internal  reviews.  A  majority  of  studies  reviewed  had 
significant  shortfalls  of  the  kind  that  could  have  been  caught  by 
a  reasonable  internal  "murder  board"  process.  Possible  reasons 
for  the  shortfalls  are:  lack  of  a  study  advisory  group,  lapse  of 
time  between  study  completion  and  report  publication,  perception 
that  the  report  is  just  "for  the  record,"  interruptions  in  the 
study,  and  failure  to  consider  audiences  other  than  the  sponsor. 

A  majority  of  the  reviewed  studies  had  difficulties  because  of 
unclear  purposes  or  objectives,  or  at  least  lack  of  agreed 
understanding  of  the  objectives.  The  problem  was  aggravated  in 
one  case  by  involvement  of  several  command  levels,  each  with  a 
different  view,  which  resulted  in  an  infeasible  scope  of  work 
within  the  planned  schedule.  In  another  case,  the  study  group 
apparently  interpreted  the  problem  quite  differently  than  did  the 
sponsor,  resulting  in  an  almost  useless  product.  Another  study 
group  used  an  inappropriate  method  which  may  have  been  due  to  very 
broad  process-type  objectives  rather  than  results-type  objectives. 

All  reviewed  studies  had  serious  problems  of  validity  of  results 
because  of  small  sample  sizes,  lack  of  sensitivity  analyses  and 
failure  to  estimate  confidence  ranges.  These  problems  occur 
particularly  when  gaming  or  simulation  efforts  are  involved,  where 
often  only  one  scenario  or  one  iteration  of  a  game  is  played.  The 
problem  of  validity  is  almost  always  present  in  situations  where 
broad  suggestive  analyses  are  preferred  to  narrower  definitive 
analyses . 

A  majority  of  the  reviewed  studies  had  problems  relating  methods 
of  solution  to  the  studies'  objectives.  In  some  cases,  the 
selected  measures  of  effectiveness  were  either  not  defined  or  not 
logically  related  to  the  objectives,  leading  to  doubts  about  the 
validity  of  the  conclusions.  In  one  case,  the  method  seemed 
incapable  of  measuring  factors  essential  to  the  objectives, 


causing  a  credibility  problem.  This  type  of  shortcoming  is 
related  to  that  of  validity  in  that  problems  with  both  are  caused 
by  trying  to  do  as  much  as  possible  within  limited  time  using 
whatever  methods  available. 

Reviewed  studies  involving  cost  analysis  were  judged  to  be  shallow 
in  respect  to  the  cost  analysis,  even  though  costs  were  key  to  the 
studies.  The  consequence  was  to  undermine  the  credibility  of  the 
results.  A  possible  reason  for  the  shallowness  was  the 
orientation  of  the  studies  towards  filling  requirements,  when  cost 
appears  to  be  of  secondary  importance,  rather  than  towards  helping 
with  allocation  decisions,  which  is  a  more  desirable  orientation. 

A  majority  of  the  studies  showed  incomplete  integration  of 
substudies  into  coherent  analyses.  This  was  demonstrated 
principally  by  inconsistencies  between  annexes  and  the  body  of 
reports.  In  some  cases,  the  actual  analyses,  consisting  of 
simulations  at  different  levels,  appeared  to  be  independent  and 
partly  conflicting.  As  with  the  methods  problem  above,  this 
problem  seems  to  be  a  result  of  trying  to  tackle  too  big  a  problem 
with  convenient,  but  not  necessarily  appropriate,  tools. 

Three  other  lessons  of  particular  importance  came  from  the 
reviews.  First,  one  study  was  rendered  almost  useless  because  the 
study  group  failed  to  consider  the  changing  nature  of  the 
requirements  that  were  driving  the  study.  Although  consideration 
of  change  might  be  assumed  as  normal  for  a  good  study  group,  it 
cannot  be  taken  for  granted.  The  second  special  lesson  is  that  of 
a  sponsor  who  wrote  an  executive  summary  which  was  inconsistent 
with  the  body  of  the  report.  The  sponsor  also  apparently  modified 
the  study's  caveats  concerning  implementation  of  the  results. 
Readers  are  thus  not  only  confused,  but  also  have  their  confidence 
in  the  results  destroyed.  Finally,  a  sponsor  made  a  commitment 
to  provide  documentation  which,  in  fact,  was  not  available  to 
allow  the  study  to  proceed  on  schedule.  Such  events  almost 
completely  undermine  the  Army's  positions  on  contracts  and  often 
cause  serious  loss  of  time  and  professional  staff  effort  on  in- 
house  studi es . 


COMMENT 

It  is  important  to  recognize  that  the  peer  review  process  is 
designed  to  deliberately  tease  out  difficulties  and  problems 
related  to  study  accomplishment  so  that  the  study  community 
learns  from  the  mistakes.  The  overall  objective  is  the 
improvement  of  the  quality  of  studies.  Thus,  although  many 
examples  of  problems  are  identified  here,  a  balanced  perspective 
must  be  maintained.  Study  performers  and  sponsors  are  to  be 
commended  for  what  they  do.  The  studies  highlighted  in  the 
preceding  pages  exemplify  what  is  good  about  the  study  program. 


REVIEW  OF  ARMY  ANALYSIS  EXTENDED  (RAAEX) 


The  purpose  of  the  Review  of  Army  Analysis  Extended  (RAAEX)  was  to  improve  the 
contribution  made  by  analysis  in  illuminating  and  solving  issues  and  problems 
of  interest  to  the  Army.  This  review  was,  in  a  sense,  an  extension  of  the 
19/8  Review  of  Army  Analysis.  The  overall  task  was  to  assess  the  Army's 
current  analysis  system  and  its  uses  and  to  propose  improvements  in  policy, 
procedure,  programs  and  organizations.  In  general,  the  group  sought 
improvements  that  could  be  achieved  by  better  management  practices  and  by 
exploiting  state-of-the-art  technology  in  telecomnunications  and  automated  data 
processing. 

The  RAAEX  study  team  was  co-chaired  by  the  Deputy  Under  Secretary  of  the  Army 
(Operations  Research)  and  the  Technical  Advisor  to  the  Deputy  Chief  of  Staff 
for  Operations  and  Plans.  The  remainder  of  the  study  group  comprised 
representatives  of  those  elements  of  the  HQDA,  DA  FOA,  AMC  AND  TRADOC  that  are 
involved  directly  in  Army  studies  and  analyses  or  which  routinely  employ 
operations  research  techniques  in  support  of  their  work.  The  initial  RAAEX 
meeting  was  on  9  July  and  reports  were  completed  on  31  August  1984. 

The  work  of  the  RAAEX  study  team  addressed  seven  basic  study  objectives. 

•  The  first  was  to  clarify  definitions  of  studies  and  analyses  and  provide 
for  appropriate  of  management  of  them. 

t  The  second  study  objective  was  to  assess  the  extent  to  which  the  actions 
taken  as  a  consequence  of  the  1978  Review  of  Army  Analysis  have  benefitted 
the  Army. 

•  The  remaining  five  objectives  concern  identification  of  appropriate 
actions  to  improve  the  following: 

•  Problems  Selected  for  Study  and  Analysis  -  The  Army  analysis 
community  should  work  ainly  on  problems  whose  solutions  would 
be  of  high  benefit  to  the  Army. 

•  Quality  of  Work  -  Army  analyses  should  be  pertinent,  consistent, 
va  lid,  and  credible. 

t  Productivity  -  Army  analyses  should  be  efficiently  conducted  and 
resources  should  be  adequate. 

•  Organizational  Arrangements  -  The  Army  analysis  community  should  be 
organized  to  facilitate  efficient  conduct  of  an  integrated  program  of 
studies,  to  provide  proper  guidance  and  control  of  studies  and 
analyses,  to  encourage  coordination  of  related  study  activities  and 
to  minimize  analysis  gaps  and  needless  overlaps. 

•  Support  to  Army  in  the  Field  -  The  Army  analysis  community  should 
provide  support  to  the  functions  of  training,  planning  and 
operations. 


From  the  study  objectives,  sixteen  specific  tasks  were  developed  and  assigned. 
Individual  reports  were  reviewed  in-process;  final  reports  contained  findings 
and  recommendations.  From  these,  nearly  1UU  RAAEX  recommendations  emerged. 
They  can  be  arranged  logically  into  five  broad  groups. 

•  Study  Program  Management  -  Clarify  definitions  of  study  and  analysis  with 
provisions  for  appropriate  degrees  of  management,  improve  ability  to 
define  major  Army  issues  and  achieve  better  program  balance. 

•  Study  and  Model  Integration  -  Develop  a  top  down  driven  Army-wide  Mission 
Area  Analysis  process  to  provide  greater  horizontal  and  vertical 
integration  of  force  and  combat  developments.  Reaffirm  commitment  to  the 
hierarchy  of  models  and  the  Army  Model  Improvement  Program. 

•  Quality  of  Analysis  -  Improve  policy  and  procedures  for  assuring  quality 
of  Army  analysis,  emphasize  analysis  research  efforts  to  provide  for 
growth  in  future  capability,  and  improve  management  of  the  professional 
development  of  military  and  civilian  operations  research  analysts. 

•  Functional  Support  -  Increase  analysis  support  to  the  Army  in  the  field, 
increase  capability  for  conducting  analysis  of  manpower  and  personnel 
issues  and  logistic  issues. 

•  Analysis  Interfaces  -  Increase  interaction  with  analysis  activities 
external  to  the  Army,  increase  integration  of  testing  and  analysis, 
strengthen  interface  between  cost  analysis  and  other  Army  analysis,  and 
improve  procedures  for  providing  essential  vulnerability  and  lethality 
input  data. 


14 


,  -V  *. 

r 

» 


I 


END 

FILMED 

2-85 

DTIC 


